The Cross-Cultural Validity of the Moral Judgment Test: Findings from 29 Cross-Cultural Studies

The Moral Judgment Test (MJT) was developed 30 years ago to assess simultaneously moral attitudes and moral judgment competence for use in evaluation and cross-cultural studies (Lind, 1978; 1985a; in press). Rigorous criteria for checking the pragmatic validity and crosscultural validity of the MJT were drawn from well-established postulates of cognitive-developmental theory: a) Preference hierarchy (Rest), b) Affective-cognitive parallelism (Piaget), and c) Quasi-simplex structure (Kohlberg). Research is presented showing that the MJT has been successfully validated in 29 different language versions since its conception. Research also corroborates Piaget`s and Kohlberg’s notion that morality has a competence-aspect which can be clearly distinguished from, and measured independently of, its affective aspects (like moral orientations or attitudes or values). In sum, the MJT is well suited for domestic as well as cross-cultural research into moral development and education. Cross-Cultural Validity of the Moral Judgment Test 3 Though he was not the first to use this concept. Already in 1874 in “The descent of man, and selection in 3 relation to sex”, Charles Darwin wrote on moral competencies in the way we use this term today. Three decades ago, most measurement in the domain of moral judgment and behavior are confined to assessing moral affects, that is, the preference for principled moral reasoning or the importance of various moral orientations. Kohlberg’s Moral Judgment Interview (MJI) appeared to assesses also the cognitive aspect, i.e., moral judgment competence (Colby et al., 1987; Kohlberg, 1958; 1964; 1981). Yet this method is cumbersome and too costly to be used in research and evaluation studies which require large samples. Moreover, the MJI provides only one combined score for both aspects (Lind, 1989). While there has been a long tradition of measuring affects (like orientations, attitudes and values) in the moral domain, the measurement of competencies in the moral domain ha been given little attention. Up until the work of Piaget and Kohlberg, psychologists had not even be aware of the fact that moral behavior has a competence aspect and, therefore, confined morality solely to the affective domain of human behavior (Krathwohl et al., 1962). Kohlberg (1964) was the first to provide an explicit definition of moral judgment competence, as "the capacity to make decisions and judgments which are moral (i.e., based on internal principles) and to act in accordance with such judgments" (p. 425; emphasis added). Note that this 3 definition refrains from imposing specific moral values on each and every individual but requires only that each individual – regardless of culture – pursues her or his moral values in a morally consistent manner. This ensures cultural fairness when measuring moral judgment competence. Therefore, 30 years ago, Lind and his associated set out to develop a new instrument, the Moral Judgment Test (MJT), to assess both aspects of moral judgment behavior simultaneously: moral orientations (the affective aspect) and moral judgment competence (the cognitive Cross-Cultural Validity of the Moral Judgment Test 4 Note that the MJT has been constructed only for use in scientific research and evaluation studies (e.g., for 4 evaluating the effects of certain methods of moral or character education, but not for diagnosing or selecting individuals or group of individuals. The MJT is not suited for the latter use, and as the author, I do not approve of it. A copy of the MJT may be obtained from the author: http://www.uni-konstanz.de/ag-moral/ . As Kohlberg (1985) notes: “In studying moral behavior we are concerned with studying action in which 5 the subject gives up something or takes risks where not doing so would appear to be to his or her immediate advantage. ... Thus, it is the overcoming of these situational pressures on either a verbal or a physical level that constitutes the test of moral behavior” (p. 522). aspect) to be used in large scale cross-cultural and longitudinal studies (Lind 1978; 1985a; in press; Lind & Wakenhut, 1985). The biggest difficulty was to define a moral task. The 4 measurement of a particular competence it closely tied to the kind of tasks for which that competence is needed. Lind and his colleagues considered several options (Lind, 1978; 1985a; Lind & Wakenhut, 1985). To test moral judgment competence as defined by Kohlberg (1964), it would be not sufficient to observe, as in rule-conformity research, merely a participant’s ‘ability’ to obey the rules set up by society (cf., May & Hawthorne, 1928; Milgram, 1974). Informed by the theory of communicative ethics (Habermas, 1983; Apel, 1990), by Piaget’s use of “counter-suggestions” in his clinical interviews (see also Lourenço & Machado, 1996; Inhelder et al., 1974) and by Keasey’s (1974) research on adolescents’ ability to deal with counter-arguments, they chose as a moral task a communication situation in which the participants had to rate moral arguments pro and contra their own opinion on a specific moral issue. That is, for the participant the main moral task involved in taking the MJT is to engage in a moral discourse by rating arguments speaking in favor and against her or his opinion on a fundamental moral dilemma. Counter-arguments often elicit both self-protective emotions as well as moral emotions (Haidt, 2001), that is, the tendency to protect one’s own judgment, and the tendency to seek moral truth as Festinger’s (1957) theory of cognitive dissonance Cross-Cultural Validity of the Moral Judgment Test 5 As Kohlberg (1985) notes: “In studying moral behavior we are concerned with studying action in which 6 the subject gives up something or takes risks where not doing so would appear to be to his or her immediate advantage. ... Thus, it is the overcoming of these situational pressures on either a verbal or a physical level that constitutes the test of moral behavior” (p. 522). Semantic problems refer to problems of word and sentence meaning in a rather direct sense of "how 7 speakers know that a given utterance represents a given thought or idea. Thus semantics is concerned with a type of translation: the translation from thought into utterance, and vice versa" (Wojcik, 1998). describes it. As much research has shown (e.g., Keasey, 1974; Damasio, 1994), it takes high judgment competencies in order to control strong moral and amoral emotions. Pragmatic and Cross-Cultural Validity of Psychological Tests In an objective test of moral judgment behavior like the MJT, complex moral thoughts had to be represented in only one short sentence in order not to overburden the subjects’ short term memory with long and complex statements when making the evaluative response. In contrast to an open-ended interview in an objective test, the subject is also deprived of the possibility of making clarifications and explanations to make sure that the test scorer truly understands what she or he wants to communicate. Therefore, objective tests must not only be checked carefully for semantic validity but also for pragmatic validity if we want to be sure that there is maximum communicative validity or, as Campbell (1963) called it, “conceptual overlap.” By pragmatic validity of a test we mean that each item of a test and the test as a whole 7 correctly represents what we want it to mean. This requires also semantic validity. Yet it is more than that. The semantic validity of the German master version of the MJT has been checked by carefully re-reading the items and by asking six noted experts in the area of moral Cross-Cultural Validity of the Moral Judgment Test 6 I wish to thank all experts involved in this: Tino Bargel, Rainer Döbert, Thomas Krämer-Badoni, Gertrud 8 Nunner-Winkler, Gerhard Portele, and Roland Wakenhut. development research to critically examine each item. In the process of cross-cultural vali8 dation of the MJT, we sought to establish semantic validity through three means. First, the author of the foreign language version, typically herself or himself an expert on Kohlberg’s stage model and Lind’s dual aspect theory of moral behavior and development, not only translates the test item by item but also checks her or his translations against the theory. Second, most authors double-check on semantic validity through back translations. Third, in cases of uncertainty, experts on the theory who also are native speakers of the target language are asked to critically comment on the items of the MJT. Pragmatic validity refers not only to the semantic correctness of a statement but also to its emotional and practical meaning. Our instructions and test items may be hundred percent semantically correct and still the subjects may understand them in a different way and thus their responses may be easily misinterpreted and mis-scored. We must be all the more alert when we deal with complex matters like morality. As Kohlberg (1958, 1981) warned, a single argument, taken by itself, can never be a reliable sign of a specific moral orientation or moral judgment competence; it must be always interpreted in context. This is why traditional ways of checking the “reliability” and “validity” of test items are insufficient if not just wrong. When studying the pragmatic validity of a test, we need to look at relationships and structures. The semantic validity of the master version of the MJT was checked in two ways. First, a small sample of subjects was asked to talk aloud when filling out the MJT and write down any comments they wanted to make. From this material we could detect many misunderstandings which were provoked by the wording of the MJT. Subsequently, the test items were revised Cross-Cultural Validity of the Moral Judgment Test 7 In older publications Lind also used correlation of moral judgment competence with level of education as 9 a fourth criterion. Because there is a certain circularity involved in this criterion, it is dropped from this list. "However, one should note that there are cognitive aspects to all of Rest's components, and Kohlberg's 10 idea of a stage as a structured whole or a world view cuts across Rest's componential model. ..." (Higgins, 1995, p. 53). and resubmitted to the same procedure. Second, the responses of the subjects were submitted to four types of relational analysis, which will be described below in more detail. Looking at the relations and inter-correlations between the responses to the MJT, enables one to detect further instances of pragmatic invalidity, which the subjects did not need to be aware of. This latter method is also used to check on the pragmatic equivalence of translated versions of the MJT. Signs of pragmatic equivalence 9 The pragmatic validity and cross-cultural equivalence of the MJT can be checked by using three well-established facts regarding the nature of moral judgment behavior (Lind 1985 a; 1985 b; 2002; in press; Schillinger, 2006): 1. Two aspects: Morality is not only a matter of moral orientations (attitudes, values etc.) but also of moral judgment competence (Kohlberg, 1963). Both aspects cannot be separated but can be clearly distinguished (Piaget, 1976; 1981; Higgins, 1995). While it would be easy 10 to simulate one’s moral orientations in any direction, it should not be possible to simulate one’s moral judgment competence upward (Emler et al., 1983; for confirmation, see Lind, 2002). Cross-Cultural Validity of the Moral Judgment Test 8 Quasi-simplex is a simplex in which sizable error measurements are allowed. Perfect simplex is reason11 able only if measurement errors are negligible. I prefer the weaker prediction, though one could also argue otherwise. I wish to thank Debbie D. Reese for this clarification. 2. Quasi-simplex: Moral orientations do form a stage-typical order. Stage-typical orientations which are perceived as adjacent (e.g., stage 2 and 3 orientations in Kohlberg’s stage model) are rated in a more similar way than moral orientations which are considered as being more distant (e.g., stage 2 and stage 5 orientation). Thus moral stage orientations should, as Kohlberg (1958) argued, correlate higher the closer they are located on Kohlberg’s scale. They should form a Quasi-Simplex Structure. 11 3. Preference order: Stage-typical moral orientations form a universal hierarchical preference order (Kohlberg 1958, 1984). Thus, regardless of cultural and ideological background, social class, age or gender, people should prefer (or reject) them in the same way (see Rests, 1969; Narvaez, 1998, p. 14). 4. Parallelism: According to Piaget’s theory of Affective-Cognitive Parallelism, measure of moral orientation and moral competence should correlate strongly with each another (Lind 1978; 2002). However, this correlation may break down if something is at stake for the participant because he or she may then simulate the orientation measures in the direction of greater ‘social desirability.’ Cross-Cultural Validity of the Moral Judgment Test 9 The first criterion, supported by two laboratory experiments (Lind, 2002; Wasel, 1994) is not required as 12 a criterion for cross-cultural validation for economical reasons. Note that Kohlberg used relative frequency of stage usage in interviews as an indicator of stage pre13 ference (they are also called ipsative because they always add up to 100 percent), implying that some indices must correlate negatively with one another (as some percentages get high, others must go down by definition). Originally, Kohlberg (1958) defined six stages of moral orientations, which he later reduced to five, but 14 afterwards reconfirmed. See Kohlberg, Boyd & Levine, 1990. Up to the year 2001, the test asked respondents to rate the arguments’ “degree of acceptability.” To 15 emphasize more the subjectivity of this rating task, the subject is now instructed to express how much she or he accepts or rejects each argument. I wish to thank Dr. Michael Hauan, University of Missouri, for convincing me on this issue. Findings on the cross-cultural validity of the MJT The last three criteria have been used as validation criteria in cross-cultural research. In sum, 12 these are the findings: 1. Quasi-simplex structure: In his original study of 83 boys age 10 to 16, Kohlberg (1958, pp. 100 & 104) found a pattern of correlations forming such a simplex-structure as Figure 2 shows. Comparing Kohlberg’s data with an ideal simplex-structure from fictitious data (Figure 1) shows that they fit well though not perfectly. 13 Studies using the MJT show an even better fit to this criterion (for an example, see Figure 3). The quasi-simplex structure was found in all cultures in a similar way. In no study was this hypothesis disconfirmed. 2. Hierarchical preference order for Kohlberg’s six stages of moral orientations: Moral 14 attitude are defined as the subject’s mean acceptability ratings of all arguments that repre15 sent a particular moral orientation. Thus, attitudes toward each stage are represented either by an index ranging from -16 to +16 or, if means are calculated, by an index from -4 to +4. In order to be valid, the attitudes toward the six Kohlbergian stages must be ordered according to their stage numbers, with the highest stage 6 preferred the most and stage 1 preferred the least. Cross-Cultural Validity of the Moral Judgment Test 10 Note, however, that this is true only for observations in “regular” situations. Affective-Cognitive Paral16 lelism seems to become unobservable if something is at stake for the subject (Lind, 2002a; 2002b). Subjects may simulate socially desirable moral attitudes or may “underachieve” by showing less moral judgment competence than they are capable of, or do both, thus blurring the picture we get. While indices of moral attitudes are susceptible to As Figure 1 shows, this is indeed the case for the samples from various countries. In fact, all 29 language-versions of the MJT show the same monotonous increase of preference from low stage orientations to high. In all cultures, principled moral orientations (Stages 5 and 6) are preferred most as adequate level of reasoning for solving moral dilemmas, and preconventional orientations (Stages 1 and 2) are rejected the most. As expected, the preferences for conventional orientations (Stage 3 and 4) are in between these extremes. Although, the preference for conventional reasoning remains always within this range, it varies considerably from culture to culture. Participants in more traditional cultures prefer the norms of the social group to which they belong more than participants from more modern cultures do. 3. Affective-cognitive parallelism: This criterion is operationalized as the pattern of correlations between the six moral orientations on the one side and the C-index for moral judgment competence on the other. The parallelism criterion is also very well met by all language-versions of the MJT. In Figure 5, the findings from a study of German 1 semester university students are depicted as st an example (for more examples, see also Lind, 1985; 1985a). In all cultures, the MJT’s Cindex correlates systematically with the subjects’ attitudes towards each of the six stagetypical moral orientations: it correlates highly negative with low-stage-orientations and highly positive with high-stage-orientations, while the correlations with medium stage-orientations are in between. In other words, the higher the moral judgment competence of people, the more clearly they reject low stage moral reasoning as inadequate, and the more clearly they prefer stages 5 and 6 as adequate stages of reasoning and discourse for solving a moral dilemma. Cross-Cultural Validity of the Moral Judgment Test 11 simulation either “up” or “down” (Emler et al., 1983), moral competencies may not fully show if the situation is aversive as, e.g., under time pressure. Study by Latif Anwar, Faculty of Education Study, Unversiti Putra Malaysia 43400 Serdang Selangor 17 Malaysia, personal communication. Again, this holds true for all different language versions of the MJT. As an example, the findings from a Malaysian study are depicted in Figure 6.